I'm new to processing but have been trying to pick it up lately. I am a painter and know next to nothing about code but I did successfully use a code this month to write video to LEDs. I am now really interested in the possibility of being able to abstract a video by assigning pixels to certain areas of a screen. I'd like to be able to sort the pixels of each frame of a video by color so that each frame still contains its original pixels but not in their original order. Any chance theres a code that will somehow do this for me out there? :)

Currently trying to consult people in the engineering school of my university but they're not very familiar with processing.

Anything would help, thanks !!

PImage img;
color c = color(random(255), random(255), random(255));

void setup() { 
  size(400, 400);
  img = createImage(400, 400, RGB); 
  img.loadPixels();
  img.updatePixels();
  image(img, 0, 0);
}

void draw() {
  myRectangle(10, 10, 40, 50);
}

void myRectangle(int xPos, int yPos, int widthRect, int heightRect) {
  img.loadPixels();
  for (int y = yPos; y < yPos + heightRect; y++) {
    for (int x = yPos; x < xPos + widthRect; x++) {
      img.set(y*width, x, c);
    }
  }
  img.updatePixels();
}

Now I want to use this values to set the pixels colors of a RGB LED strip.

The problem is that when I separate and read (at the processing console) de values of the color for each pixel, I get almost all my pixels with 0 for red... But the squares drawn in the screen seems OK, only the values I try to save to the matrix for latter use that seems odd.

Below is my code ( with thanks to bourdonprince and GoToLoop):

import processing.video.*;
Capture webcam;

import processing.serial.*;
Serial myPort;

float[][] rx;
float[][] gx;
float[][] bx;

void setup() {
  size(440, 320);

  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);

  rx = new float[16][15];
  gx = new float[16][15];
  bx = new float[16][15];

  webcam = new Capture(this, 320, 240);  
  //String[] devices = Capture.list();
  //println(devices);
  webcam.start();
}

void draw() {

  if (webcam.available() == true) {
    webcam.read();
    image(webcam, 0, 0, width, height);
    saveFrame("/data/image.jpg");
  }

  delay(100);

  PImage img = loadImage("image.jpg");
  int detail = 30; // tamanho do quadrado

  int count_i = 0;
  int count_j = 0;


  for (int j=0; j< height; j+=detail) {
    
    count_i = 0;

    for (int i=0; i < width; i+=detail) {
      PImage newImg = img.get(i, j, detail, detail);
      noStroke();
      color cor = extractColorFromImage(newImg);
      fill(cor);
      rx[count_i][count_i]= red(cor);
      gx[count_i][count_j]= green(cor);
      bx[count_i][count_j]= blue(cor);

      count_i++;
    }
    count_j++;
  }


  for (int i=0; i<15; i++)
  {
    for (int j=0; j<11; j++)
    {
      println((i) + "," + (j) + "," + int( rx[i][j] )+ "," + int (gx[i][j]) + "," + int (bx[i][j]));
      //delay(100);
      myPort.write((i) + "," + (j) + "," + int( rx[i][j] )+ "," + int (gx[i][j]) + "," + int (bx[i][j])  + "\n");
      delay(50);
    }
  }

  delay(2000);
}



//função para extrair a cor média em um recorte
color extractColorFromImage(PImage img) {
  img.loadPixels();
  color r = 0, g = 0, b = 0;

  for (final color c : img.pixels) {
    r += c >> 020 & 0xFF;
    g += c >> 010 & 0xFF;
    b += c        & 0xFF;
  }

  r /= img.pixels.length;
  g /= img.pixels.length;
  b /= img.pixels.length;

  return color(r, g, b);
}

``

import SimpleOpenNI.*;

SimpleOpenNI kinect;

int closestValue; int closestX; int closestY; int Threshold; int PixelsInFrame;

void setup() {

size(640, 480); kinect = new SimpleOpenNI(this);

kinect.enableDepth(); //kinect.enableRGB(); }

void draw() {

closestValue =800; Threshold = 1000;

kinect.update();

int[] depthValues = kinect.depthMap();

//int currentDepthValue =

//PImage depthImage = kinect.depthImage();

// get the depth array from kinect

for (int y = 0; y < 480; y++) { for (int x = 0; x < 640; x++) {

  int i = x + y *640;

  int currentDepthValues  = depthValues[i];


  //Find the pixels in certain distance bound)
  //fIND THE PIXELS IN Y AND X AXIS and paint them one colour
  //find the pixels and make them interact with other shapes

  if ( currentDepthValues > 0 && currentDepthValues <= Threshold && currentDepthValues >= closestValue) {

  currentDepthValues = PixelsInFrame;

  print(PixelsInFrame);



    /*loadPixels();
     for (int k = 0; k < depthValues[i].length; k++){
     pixels[k] = pink; 
     }

     updatePixels();*/


    //depthValues[i] = color(0, 255, 0);
  }
}

}

image(kinect.depthImage(), 0, 0);

}

``

Here is a simple code. What is wrong with these calculations ? Why aren't the rc,gc,bc values getting closer to current frame values ?

import processing.video.*; 
import processing.serial.*; 

Capture video;
Serial myPort; 

void setup() 
{
  size(640, 480);
  video = new Capture(this, width, height, 30);
  video.start();
}

void captureEvent(Capture video) 
{ 
  video.read(); 
}

void draw() 
{  
  video.loadPixels();
  image(video, 0, 0);

  loadPixels();

   float rc = 0; // Initialaizing values are 0, so at the start, strange thinks may display 
   float gc = 0; 
   float bc = 0;

  for (int x = 0; x < video.width; x++ ) 
  {
    for (int y = 0; y < video.height; y++ ) 
    {
      int loc = x + y * video.width;

      color currentColor = video.pixels[loc]; //current values, taken from cam
      float r3 = red(currentColor);
      float g3 = green(currentColor);
      float b3 = blue(currentColor);

      rc = rc*0.99 + r3*0.01;        //with each iterations, rc,gc,bc values are closer to current frame
      gc = gc*0.99 + g3*0.01;        //so after some time, these values will be almost equal to current r3, g3, b3
      bc = bc*0.99 + b3*0.01;

      pixels[loc] = color(rc,gc,bc); //I am drawing the values on screen, after some time, I should get clear a clear picture. 
    }
  }
  updatePixels();
}

import processing.video.*;

Capture video;

PGraphics pg;

PImage backgroundImage;
float threshold = 30;

void setup() {
  size(320, 240);
  video = new Capture(this, width, height);
  video.start();

  backgroundImage = createImage(video.width, video.height, RGB); 
  pg = createGraphics(320, 240);
}

void captureEvent(Capture video) {
  video.read();
}

void draw() {
  pg.beginDraw();

  loadPixels();
  video.loadPixels();
  backgroundImage.loadPixels();

  image(video, 0, 0);
  for (int x = 0; x < video.width; x++) {
    for (int y = 0; y < video.height; y++) {
      int loc = x + y * video.width;



color fgColor = video.pixels[loc];
color bgColor = backgroundImage.pixels[loc];

float r1 = red(fgColor); float g1 = green(fgColor); float b1 = blue(fgColor);
float r2 = red(bgColor); float g2 = green(bgColor); float b2 = blue(bgColor);
float diff = dist(r1, g1, b1, r2, g2, b2);


if (diff > threshold) {
  pixels[loc] = fgColor;
} else {
  pixels[loc] = color(0, 0);
}
    }}
    pg.updatePixels();
    pg.endDraw();


    saveFrame("line-######.png");
}


void mousePressed() {
  backgroundImage.copy(video, 0, 0, video.width, video.height, 0, 0, video.width, video.height);
  backgroundImage.updatePixels();
}

still trying to understand what the color values return and how to decipher them thanks

Is there a better way to manually mix ARGB?

EDIT: I found the correct way to do this. The corrected program is down a few posts ( if anyone is interested in the method).

PImage A, B,C;
boolean showMix = false;
void setup(){

     size(600,600);
     A = createImage(600,600,ARGB);
     B = createImage(600,600,ARGB);
     C = createImage(600,600,ARGB);
     A.loadPixels();
     for(int i = 0; i < 300; i++){
          for(int j = 0;j < 300; j++){
               A.pixels[i * 600 + j]= 0x7FFF0000;
          }
     }
     A.updatePixels();
     B.loadPixels();
     for(int i = 150; i < 450; i++){
          for(int j = 150;j < 450; j++){
               B.pixels[i * 600 + j]= 0x7F00FF00;
          }
     } 
     B.updatePixels();  
     mix();  

}
void draw(){
     background(255);
     if(showMix)
          image(C,0,0);
     else{
          image(A,0,0);
          image(B,0,0);
     }
}
void keyPressed(){
     showMix = !showMix;
}
void mix(){
     float ba,br,bg,bb; //background a,r,g,b
     float fa,fr,fg,fb; //foreground
     float oa, or,og,ob; //output
     int a,r,g,b;
     A.loadPixels();
     B.loadPixels();
     C.loadPixels();
     for(int i = 0; i < 600;i++){
          for(int j= 0;j<600;j++){
               int index = i * 600 + j;
               ba=A.pixels[index]>>24 & 0xFF; fa=B.pixels[index]>>24 & 0xFF;
               br=A.pixels[index]>>16 & 0xFF; fr=B.pixels[index]>>16 & 0xFF;
               bg=A.pixels[index]>>8 & 0xFF; fg=B.pixels[index]>>8 & 0xFF;
               bb=A.pixels[index] & 0xFF; fb=B.pixels[index] & 0xFF;
               ba/=255;fa/=255;      //convert alpha to 0.0 - 1.0
               oa = fa + ba *(1.0-fa);
               or = (fr * fa) + ( br*ba*(1.0-fa) );
               og = (fg * fa) + ( bg*ba*(1.0-fa) );
               ob = (fb * fa) + ( bb*ba*(1.0-fa) );
               oa *= 255;  // convert to 0-255
               a=Math.round(oa); r=Math.round(or); g=Math.round(og); b=Math.round(ob);               
               C.pixels[index]= a<<24|r<<16|g<<8|b;
          }  
     }
     C.updatePixels();
}   

I am scanning an image line by line and outputting the greyscale of each pixel to Pure Data via OSC, all of it is working ok but it seems super intensive for the computer, the fans are on full speed and sometimes Processing crashes a few minutes in.

Is there anything I can do to make this code less intensive? Might be missing something obvious?

It could be because of the frameRate, and if so are there any suggestions of other ways of doing this? So that I can still run the scanning pretty fast

Thanks in advance for any help

import oscP5.*;
import netP5.*;

NetAddress remote;
OscP5 oscP5;
//int incomingOSCData;
// int sendTestData;
// String msgType;

PImage img;
int x, y, s, r;

void setup() {
  size(500, 500);
  oscP5 = new OscP5(this,12000);
  remote = new NetAddress("127.0.0.1",3000);
  s = 10;
  stroke(255);
  reset();
  frameRate(200);
  img = loadImage("10.jpg");
  background(0);
  img.loadPixels();
}

void reset(){
  background(0);
  x = 0;
  y = 0;
  OscMessage msg = new OscMessage("/sync");
  msg.add(1);
  oscP5.send(msg,remote);
}

void draw() {
  if(x>=width){ y+=s; x=0; }
  if(y>=height) reset();
  delay(100);
  int loc = x + y*img.width;
  noStroke();
  color c = img.pixels[loc];
  // int alpha = (c >> 24) & 0xFF;
  int red   = (c >> 16) & 0xFF;
  int green = (c >> 8)  & 0xFF;
  int blue  =  c        & 0xFF;
  int gray = ((red + green + blue) / 3);
  // println(alpha + " " + red + " " + green + " " + blue);
  fill(c);
  rect(x++,y,4,4);
  // println(gray);
  OscMessage msg = new OscMessage("/grayscale");
  msg.add(gray);
  oscP5.send(msg,remote);
}

void setup() {
  size(1024,896,P2D);
  frameRate(500);
  noStroke();
  smooth(0);
}

void draw() {
  background(0);
  ellipse(mouseX,mouseY,100,100);
  if(mousePressed){
    drawPixels(256,224);
  }
  println(frameRate);
}

void drawPixels(int retroResX, int retroResY) {
  int pixelScale = width/retroResX;
  loadPixels();
  for(int y=0;y<retroResY;y++){
    for(int x=0;x<retroResX;x++){
      for(int py=0;py<pixelScale;py++){
        for(int px=0;px<pixelScale;px++) {
          pixels[((py+(y*pixelScale))*width)+(px+(x*pixelScale))] = pixels[(y*width*pixelScale)+(x*pixelScale)];
        }
      }
    }
  }
  updatePixels();
}

https://www.openprocessing.org/sketch/441474

into p5.js?

https://www.openprocessing.org/sketch/441671

// @pjs preload must be used to preload the image   
/* @pjs preload="graynoiz_128.png"; */

// Works in Processing. Does not work in Processing.js. Why  not?

PImage b;
String imageName = "graynoiz_128.png";
int shift;

public void setup() {
  size(128, 128);
  this.frameRate = 24;
  shift = 4;
  b = loadImage(imageName);
}

public void draw() {
  background(233, 233, 199);
  image(b, 0, 0);
  if (this.frameCount > 2) {
    b.loadPixels();
    color[] pix = b.pixels;
    rotateLeft(b.pixels, shift);
    b.updatePixels();
  }
}

/**
 * Rotates an array of ints left by d values. Uses efficient "Three Rotation" algorithm.
 * @param arr   array of ints to rotate
 * @param d     number of elements to shift
 */
public void rotateLeft(color[] arr, int d) {
  d = d % arr.length;
  reverseArray(arr, 0, d - 1);
  reverseArray(arr, d, arr.length - 1);
  reverseArray(arr, 0, arr.length - 1);
}

/**
 * Reverses an arbitrary subset of an array.
 * @param arr   array to modify
 * @param l     left bound of subset to reverse
 * @param r     right bound of subset to reverse
 */
public void reverseArray(color[] arr, int l, int r) {
  color temp;
  while (l < r) {
    temp = arr[l];
    arr[l] = arr[r];
    arr[r] = temp;
    l++;
    r--;
  }
}

The image loads just fine in FireFox (using MAMP). It shifts/animates fine in Processing, but not in the web page.

I want to do more complex things with the image pixels, but there seems to be some kind of error I need to resolve first, though nothing shows in the console.

Thanks,

-- Paul

I am trying to use p5.js and insisting on learning javascript - which is harder than I thought!!

Below is the Processing script I wrote that basically smears an image.

It is a fairly simple script; It finds a random pixel It gets the color of that pixel matches the following 10-100 pixels color to the original random pixel.

So somehow this explanation on get() reference about doing the same operation with pixels[] confused me a bit. https://p5js.org/reference/#/p5/get

Now, when I wrote this script and tried it, what I see is basically the original earth image and nothing happens.

the most confusing part is var instead of color I don't understand how I should specify so that my program gets the color information of that pixel instead of whatever pixels[] array is providing as an information.

Here is the original Processing Script:

import processing.pdf.*;
import java.util.Calendar;

import processing.pdf.*;
import java.util.Calendar;

PImage img;
PGraphics pg; 
boolean drawn = false;
String naming;

void setup() {
  img = loadImage("http://" + "cdni.pageflow.codevise.de/main/video_files/posters/000/000/370/v3/medium/poster-0.JPG");
  size(img.width, img.height);
  pg = createGraphics(width, height);
  pg.beginDraw();
  pg.image( img, 0, 0, width, height);
  pg.endDraw();
}

void draw() {
  pg.loadPixels();
  if(!drawn){
    int numberOfRands = 100;
  for(int i = 0; i < numberOfRands; i++){
    int r= int(random(height*width));
    color c = pg.pixels[r];
    for(int j = 0; j < random(10,100); j++){
      if(r+j < pg.pixels.length){
        pg.pixels[r+j] = c;
      }
    } 
  }
//  drawn = true;
  }
  pg.updatePixels();
  image(pg, 0, 0, width, height);
}

and here is the converted script: you can also see the same p5.js script here: https://www.openprocessing.org/sketch/438964

var img, pg;
var canvasOfset = 10;
var fillColor;
var drawn = false;

function preload() {
  img = loadImage('http://" + "cdni.pageflow.codevise.de/main/video_files/posters/000/000/370/v3/medium/poster-0.JPG');
}



function setup() {
  createCanvas(windowWidth - canvasOfset, windowHeight - canvasOfset);
  pg = createGraphics(width, height);
  pg.image(img, 0, 0, width, height);
}


function draw() {
  pg.loadPixels();
  var numberOfRands = 100;
  var d = pixelDensity();
  for (var j = 0; j < numberOfRands; j += 4) {
    var off = random(width * height);
    var fillColor = color(pg.pixels[off], pg.pixels[off + 1], pg.pixels[off + 2], pg.pixels[off + 3]);
    for (var i = 0; i < random(120, 1600); i += 4) {
      pg.pixels[off + i] = color(fillColor);
      pg.pixels[off + i + 1] = green(fillColor);
      pg.pixels[off + i + 2] = blue(fillColor);
      pg.pixels[off + i + 3] = alpha(fillColor);
    }
  }

  pg.updatePixels();
  image(pg, 0, 0, width, height);
}



function windowResized() {
  resizeCanvas(windowWidth - canvasOfset, windowHeight - canvasOfset);
}

Does anybody know why my code isn't working?

Thanks,

https://www.openprocessing.org/sketch/438530

My question relates to size(), pixelDensity() and displayDensity() ... I believe I understand them correctly, the only thing that tripped me up was the need to use pixelWidth and pixelHeight when doing pixel based operations. But it would seem to me that if I'm creating something for a "normal" display, to be projected say, then I might use something like size(1920,1080). But on a retina display like my MacBookPro I would normally use something smaller, but knowing that the quality will be the same as higher in that smaller window.

In short it doesn't seem that I can make decisions on the values for size() based upon displayDensity()? It would seem like a natural thing to want to do if developing for both retina and non-retina. Or more fundamental, I would like to be able to base the values for size() on the display pixel dimensions. Any way to do this?

So my question is, is there a faster way of drawing a byte to the screen than my approach?

Code of the drawing "algorithm":

for (int I=0; I<6144; I++){ int x,y = I/32; for(int X=0; X<32; X++) { x = X*8; // checks if the bit is == 1 and then draws the point if ( (DataIn[I] & 0x01) > 0) point(x,y); if ( (DataIn[I] & 0x02) > 0) point(x+1,y); if ( (DataIn[I] & 0x04) > 0) point(x+2,y); if ( (DataIn[I] & 0x08) > 0) point(x+3,y); if ( (DataIn[I] & 0x10) > 0) point(x+4,y); if ( (DataIn[I] & 0x20) > 0) point(x+5,y); if ( (DataIn[I] & 0x40) > 0) point(x+6,y); if ( (DataIn[I] & 0x80) > 0) point(x+7,y); } }

also sorry about my bad english

like this,,

/**
void mousePressed() {
 capture.save(“temp.jpg");
 byte[] b=loadBytes(“temp.jpg");
 server.write(b);
} 
*/

but i have no idea how to convert this byte array to PImage format in Client’s sketch.

is there any byte[] to Pimage converting method in processing???

My question today is how to randomly place a pixel after loading it. I'm not interested in manipulating it's colors just the pixels. Here is my function:

void glitch () {
  if (!glitchComplete) {

    // load the image's pixels into .pixel array 
    img.loadPixels();
    //iterate through each pixel row and column and apply the desired changes
    for (int x = 0; x < img.width; x++) {

      for (int y = 0; y < img.height; y++) {

        color pixelColor = img.pixels[y + x * img.height];
        //the formula I learned from the processing docs

        int loc = y + x * img.height;

        int randomOffset = int(random(5,20));

        //slight shift in their original location
        loc = loc + randomOffset;

        img.pixels[loc] = pixelColor;
      }
    }//end of double loop

    img.updatePixels();
    glitchComplete = true;

    // load updated image onto surface
    image(img, 0, 0, img.width, img.height);
  }

For anyone new to processing, just a heads up: " loc = loc + randomOffset;" will NOT work because this position technically doesnt exist in the pixel array. If you want to place a pixel in a random location, it must be a random location within the bounds of the pixel array (hope im using the right terminology here) .

My thinking here was I wanted to redraw the pixel at it's original location & color and simply shift it over a few pixels to produce something close to a glitch effect(instead of re-colouring the image). So can anyone suggest another method that can achieve the same results.

Your time and input is very much appreciated.

// Click the mouse to memorize a current background image
import processing.video.*;
import gab.opencv.*;

// Variable for capture device
Capture video;
OpenCV opencv;

// Saved background
PImage backgroundImage;

// How different must a pixel be to be a foreground pixel
float threshold = 20;

void setup() {
  String[] cameras = Capture.list();
  //printArray(cameras);
  size(1280, 720);
  //video.width = 1280;
  //video.height = 720;
  video = new Capture(this, 1280, 720, cameras[76]);

  video.start();

  opencv = new OpenCV(this, 1280, 720);
  opencv.startBackgroundSubtraction(5, 3, 0.5);

  // Create an empty image the same size as the video
  backgroundImage = createImage(video.width, video.height, RGB);




}

void captureEvent(Capture video) {
  // Read image from the camera
  video.read();
}

void draw() {
  if (video.available() == true) {
    video.read();
  }
  image(video, random(width),random(height));

  opencv.loadImage(video);

  opencv.updateBackground();

  opencv.dilate();
  opencv.erode();

  noFill();
  stroke(255, 0, 0);
  strokeWeight(3);
  for (Contour contour : opencv.findContours()) {
    contour.draw();

}
}

void mousePressed() {

}

** **second code to combine with first one**** **

import processing.video.*;

Capture video;
int signal = 0;

//the buffer for storing video frames
ArrayList frames;

//different program modes for recording and playback
int mode = 0;
int MODE_NEWBUFFER = 0;
int MODE_RECORDING = 1;
int MODE_PLAYBACK = 2;

int currentX = 0;

void setup() {
  size(640, 480);

  // This the default video input, see the GettingStartedCapture 
  // example if it creates an error
  video = new Capture(this, width, height);

  // Start capturing the images from the camera
  video.start();  
}

void captureEvent(Capture c) {
  c.read();

  //create a new buffer in case one is needed
  if (mode == MODE_NEWBUFFER) {
    frames = new ArrayList();
    mode = MODE_RECORDING;
  }

  //record into the buffer until there are enough frames
  if (mode == MODE_RECORDING) {
    //copy the current video frame into an image, so it can be stored in the buffer
    PImage img = createImage(width, height, RGB);
    video.loadPixels();
    arrayCopy(video.pixels, img.pixels);

    frames.add(img);

    //in case enough frames have been recorded, switch to playback mode
    if (frames.size() >= width) {
      mode = MODE_PLAYBACK;
    }
  }
}

void draw() { 
  loadPixels();

  //code for the recording mode 
  if (mode == MODE_RECORDING) {
    //set the image counter to 0
    int currentImage = 0;

    //begin a loop for displaying pixel columns
    for (int x = 0; x < video.width; x++) {
      //go through the frame buffer and pick an image using the image counter
      if (currentImage < frames.size()) {
        PImage img = (PImage)frames.get(currentImage);

        //display a pixel column of the current image
        if (img != null) {
          img.loadPixels();

          for (int y = 0; y < video.height; y++) {
            pixels[x + y * width] = img.pixels[x + y * video.width];
          }  
        }

        currentImage++;

      } 
      else {
        break;
      }
    }
  }

  if (mode == MODE_PLAYBACK) {  

    for (int x = 0; x < video.width; x++) {
      PImage img = (PImage)frames.get(x);

      if (img != null) {
        img.loadPixels();
        for(int y = 0; y < video.height; y++) {
          pixels[x + y * width] = img.pixels[currentX + y * video.width];
        }  
      }
    } 

    currentX++;

    if(currentX >= video.width) {
      mode = MODE_NEWBUFFER;
      //reset the column counter
      currentX = 0;
    }
  }

  updatePixels();
}

and I am not getting my code to work:

var video;

function setup() {
  createCanvas(320, 240);
  pixelDensity(1);
  video = createCapture(VIDEO).size(320, 240);
}

function draw() {
  background(51);

  video.loadPixels();

  //pixels from canvas
  loadPixels();
  for (var y = 0; y < height; y++) {
    for (var x = 0; x < width; x++) {
      var index = (x + y * width)*4;
      pixels[index+0] = video.pixels[index+0];
      pixels[index+1] = video.pixels[index+1];
      pixels[index+2] = video.pixels[index+2];
      pixels[index+3] = 255;      
    }
  }
  updatePixels();
}

It shows like a ghost-like video :/ http://xxnotmuchxx.tumblr.com/post/160811589716

inc = 0.01
def setup():
    size(6, 5)

def draw():
    # background(50)
    global inc
    yoff = 0
    loadPixels()
    for y in range(height-1):
        xoff = 0
        for x in range(width):
            idx = (x + y * width) * 4
            r = noise(xoff, yoff) * 255
            pixels[idx+0] = r
            pixels[idx+1] = r
            pixels[idx+2] = r
            pixels[idx+3] = 255
            xoff += inc
        yoff += inc
    updatePixels()

The error appears here: pixels[idx+2] = r

Thanks for any help!

~dallas_borealis

I'm trying to build a new image, capturing only part that interests me from another, I used this base to begin with: https://processing.org/tutorials/pixels/

I am in doubt as to the image below, the part where it seems "rainbow", would I use the function (hue), or (r, g, b) individually to capture?

And what about the circled part (very badly done, sorry) more in the middle, would have to use the function (brightness), since it is kind of "brightness" in the image?

Thanks any hint, below is the sample code, and the image as template.

Thank you,

    PImage source;       
    PImage destination;  
    float threshold = 0;

    void setup() {
      size(400, 300);
      source = loadImage("Img01.jpg"); 
      source.resize(400, 300); 
      destination = createImage(source.width, source.height, RGB);
    }

    void draw() {  
      //threshold = 127;
      threshold = 70;

      source.loadPixels();
      destination.loadPixels();

      for (int x = 0; x < source.width; x++) {
        for (int y = 0; y < source.height; y++ ) {
          int loc = x + y*source.width;
          if (brightness(source.pixels[loc]) > threshold) {
          //if (hue(source.pixels[loc]) > threshold) {
            //destination.pixels[loc]  = source.pixels[loc];  // White
            destination.pixels[loc]  = source.pixels[loc];
          }
        }
      }

      destination.updatePixels();
      image(destination,0,0);
      //image(source,0,0);
    }

    void keyPressed(){
      if (key == 'A' || key == 'a')
        threshold = threshold + 10;

      if (key == 'D' || key == 'd')
        threshold = threshold - 10;

      println(threshold);  
    }

Any and all insight/ help would be greatly appreciated!!

` import processing.video.*;

// Variable for capture device
Capture video;

// A variable for the color we are searching for.
color trackColor; 

float threshold = 25;

void setup() {
  size(640, 360);
  String[] cameras = Capture.list();
  printArray(cameras);
  video = new Capture(this, cameras[3]);
  video.start();
  // Start off tracking for red
  trackColor = color(255, 0, 0);
}

void captureEvent(Capture video) {
  // Read image from the camera
  video.read();
}

void draw() {
  video.loadPixels();
  image(video, 0, 0);

  textSize(12);
  text("pick a tool, click the color, draw yourself a friend", 10, 30); 
  stroke(#FFFFFF);

  textSize(12);
  text("press space to save your sketch", 10, 330); 
  fill(#FFFFFF);  

  // XY coordinate of closest color
  float avgX = 0;
  float avgY = 0;

  int count = 0;

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      // Using euclidean distance to compare colors
      float d = dist(r1, g1, b1, r2, g2, b2); // We are using the dist( ) function to compare the current color with the color we are tracking.

      if (d < threshold) {
        avgX += x;
        avgY += y;
        count++;
      }
    }
  }

  // We only consider the color found if its color distance is less than 10. 
  // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
  if (count > 0) { 
    avgX = avgX / count;
    avgY = avgY / count;

    // Draw a circle at the tracked pixel
    fill(trackColor);
    noStroke();
    ellipse(avgX, avgY, 8, 8);
  }
}

void mousePressed() {
  // Save color where the mouse is clicked in trackColor variable
  int loc = mouseX + mouseY*video.width;
  trackColor = video.pixels[loc];
}

void keyPressed() {
  if (key == ' ') { //space bar
  saveFrame("picture-####.jpg");
  }
}

`

    import processing.video.*;
    import java.awt.Frame;
    import java.awt.Image;
    import java.text.*;

    Capture cam;
    Letter[][] drops;
    int dropsLength;
    PImage prevFrame;
    int sWidth = 1280;
    int sHeight = 720;

    String inputString = "Įsivaizduokime pasaulį, kur visi viską žino tiksliai, ir niekada neklysta. Niekam nekiltų abejoių, koks bus rytoj oras, kaip išsaugoti tirpstačius ledynus, ar koks visatos dydis. Žvelgiant į krentantį kamuoliuką kiekvienas galėtų pasakyti: -  O šito kamuoliuko kritimo greitis 6,325 m/s. -  Tikrai taip - atsakytų kitas. Ir viskas, daugiau nebebūtų jokių diskusijų, ieškojimų, matavimų. Su absoliučiu žinojimu gyvenimas taptų nebeįdomus, monotoniškas, tokiu atveju net progresas neįmanomas. Kai pradedu taip galvoti, džiaugiuosi nežinojimu, diskusijų galimybe, tiesos ieškojimu. Klaida suvokiama kaip neišvengiamas procesas teisybės ieškojime leidžia drąsiai žengti į praktikos sritį, nebijoti suklysti, o neteisingus procesus paversti progresu, žingsneliu link tikslo. Menininkas nebėra tas genijus, kuris turi sukurti kažką naujo, tarsi nežemiško, keičiančio visą mūsų suvokimą. Jo praktikos esmė eksperimentuoti ir klysti, organizuoti jau esamomis reikšmėmis ir kurti nau";
    char[] inputLetters;
    int dupStrings = 3;   //times to dublicate text
    int k = 800;
    float threshold = 50;

    void settings() {
      size(1280, 720);
    }
    void setup() {
      String[] cameras = Capture.list();

      drops = new Letter[dupStrings][inputString.length()];  //[inputString.length()]; 

      int wspace = 50;
      inputLetters = new char[inputString.length()]; 
      splitString();


      // first row hight
      int addLineHeight = 30;                                                       
      for (int i = 0; i < dupStrings; i++) {     
            for (int j = 0; j < inputLetters.length; j++) {   
                 if  (inputLetters[j] < k + wspace ){
                   Letter testLetter = new Letter(inputLetters[j]);
                   testLetter.x = wspace;
                   testLetter.y = addLineHeight;
                   drops[i][j] = testLetter;
                   wspace += 10;   //spaces between letters

                   // new row
                   if (wspace >= sWidth) {   
                                wspace = 10;
                               addLineHeight += 40; } // space between 

                    }
                           else {
                               addLineHeight += 50;
                                }
                                                                  }
                                               }
    // cam conect//

      if (cameras.length == 0) {
        println("There are no cameras available...");
        size(400, 400);
        exit();
      }
      else {
        cam = new Capture(this, sWidth, sHeight);
        cam.start();
        cam.loadPixels();
        prevFrame = createImage(cam.width, cam.height, RGB);

        size(sWidth, sHeight);
      }
      dropsLength = inputString.length();
    }

    void captureEvent(Capture cam) {
      // Save previous frame for motion detection!!
      prevFrame.copy(cam, 0, 0, cam.width, cam.height, 0, 0, cam.width, cam.height); 
      // Before we read the new frame, we always save the previous frame for comparison!```
      prevFrame.updatePixels();  // Read image from the camera
      cam.read();
    }


    void splitString() { 

      for (int i = 0; i < inputString.length() ; i++) {
        inputLetters[i] = inputString.charAt(i);
      }
    }

    void draw() {

        loadPixels();
      cam.loadPixels();
      prevFrame.loadPixels();
        // Begin loop to walk through every pixel
      for (int x = 0; x < cam.width; x ++ ) {
        for (int y = 0; y < cam.height; y ++ ) {

          int loc = x + y*cam.width;            // Step 1, what is the 1D pixel location
          color current = cam.pixels[loc];      // Step 2, what is the current color
          color previous = prevFrame.pixels[loc]; // Step 3, what is the previous color

          // Step 4, compare colors (previous vs. current)
          float r1 = red(current); 
          float g1 = green(current); 
          float b1 = blue(current);
          float r2 = red(previous); 
          float g2 = green(previous); 
          float b2 = blue(previous);
          float diff = dist(r1, g1, b1, r2, g2, b2);

          // Step 5, How different are the colors?
          // If the color at that pixel has changed, then there is motion at that pixel.
          if (diff > threshold) { 
            // If motion, display black
            pixels[loc] = color(0);
          } else {
            // If not, display white
            pixels[loc] = color(255);
          }
        }
      }

       //Responding to the brightness/color of the screen
       for (int i = 0; i < dupStrings; i++) {
        for (int j = 0; j < dropsLength; j++) {

          if (drops[i][j].y < sHeight && drops[i][j].y > 0) {   
           int loc = drops[i][j].x + ((drops[i][j].y)-1)*sWidth;                                           
           float bright = brightness(cam.pixels[loc]);                                                                                                                                                          
            if (bright > threshold) {   
              drops[i][j].dropLetter();
              drops[i][j].upSpeed = 1;
            }
            else {                                                                            
              if (drops[i][j].y > threshold) {
                int aboveLoc = loc = drops[i][j].x + ((drops[i][j].y)-1)*sWidth;
                float aboveBright = brightness(cam.pixels[aboveLoc]);
                if (aboveBright < threshold) {                                              
                  drops[i][j].liftLetter();
                  drops[i][j].upSpeed = drops[i][j].upSpeed * 5;                           
                }
              }
            }
          }
          else {
            drops[i][j].dropLetter(); 
          }

          drops[i][j].drawLetter();
          cam.updatePixels();
        }
      }
    }

    class Letter {
      int x;
      int y; 
      int m;
      char textLetter;
      int upSpeed;
      int alpha = 150;
      Letter(char inputText) {
        x = 100;
        y = 100;
        textLetter = inputText;
        textSize(16);
        upSpeed = 1;
      }
      void drawLetter() {
       // if ( m < 1) {
        fill(150, 150, 150 , alpha);                 
        text(textLetter, x, y);

      }

      void letterFade() {
        alpha -= 5;
        if(alpha <= 0) {
          y = int(random(-350, 0));
          alpha = 255; 
        }
      }


      void dropLetter() {
      //  y++;                         
        if (y > 730) {
          letterFade();
        }
      }

      void liftLetter() {
        int newY = y - upSpeed;       
        if (newY >= 0) {
          y = newY;
        }
        else {
          y = 0;
        }
      }
    }

I'd really appreciate some help on this, I'm using the KinectPV2 library which is great but very poorly documented.

With reference to the example "MapDepthToColor" copied below I'm trying to retrieve the depth for each RGB pixel. I'm having a hard time decrypting what's going on, can someone help me out?

Thanks in advance, Charles

/*
Thomas Sanchez Lengeling.
http://codigogenerativo.com/

KinectPV2, Kinect for Windows v2 library for processing

Color to fepth Example,
Color Frame is aligned to the depth frame
*/

import KinectPV2.*;

KinectPV2 kinect;

int [] depthZero;

//BUFFER ARRAY TO CLEAN DE PIXLES
PImage depthToColorImg;

void setup() {
  size(1024, 848, P3D);

  depthToColorImg = createImage(512, 424, PImage.RGB);
  depthZero    = new int[ KinectPV2.WIDTHDepth * KinectPV2.HEIGHTDepth];

  //SET THE ARRAY TO 0s
  for (int i = 0; i < KinectPV2.WIDTHDepth; i++) {
    for (int j = 0; j < KinectPV2.HEIGHTDepth; j++) {
      depthZero[424*i + j] = 0;
    }
  }

  kinect = new KinectPV2(this);
  kinect.enableDepthImg(true);
  kinect.enableColorImg(true);
  kinect.enablePointCloud(true);

  kinect.init();
}

void draw() {
  background(0);

  float [] mapDCT = kinect.getMapDepthToColor(); // Length: 434,176

  //get the raw data from depth and color
  int [] colorRaw = kinect.getRawColor(); // Length: 2,073,600

  //clean de pixels
  PApplet.arrayCopy(depthZero, depthToColorImg.pixels);

  int count = 0;
  depthToColorImg.loadPixels();
  for (int i = 0; i < KinectPV2.WIDTHDepth; i++) {
    for (int j = 0; j < KinectPV2.HEIGHTDepth; j++) {

      //incoming pixels 512 x 424 with position in 1920 x 1080
      float valX = mapDCT[count * 2 + 0];
      float valY = mapDCT[count * 2 + 1];

      //maps the pixels to 512 x 424, not necessary but looks better
      int valXDepth = (int)((valX/1920.0) * 512.0);
      int valYDepth = (int)((valY/1080.0) * 424.0);

      int  valXColor = (int)(valX);
      int  valYColor = (int)(valY);

      if ( valXDepth >= 0 && valXDepth < 512 && valYDepth >= 0 && valYDepth < 424 &&
        valXColor >= 0 && valXColor < 1920 && valYColor >= 0 && valYColor < 1080) {
        color colorPixel = colorRaw[valYColor * 1920 + valXColor];
        //color colorPixel = depthRaw[valYDepth*512 + valXDepth];
        depthToColorImg.pixels[valYDepth * 512 + valXDepth] = colorPixel;
      } 
      count++;
    }
  }
  depthToColorImg.updatePixels();

  image(depthToColorImg, 0, 424);
  image(kinect.getColorImage(), 0, 0, 512, 424);
  image(kinect.getDepthImage(), 512, 0);

  text("fps: "+frameRate, 50, 50);
}

The minimum amount of code I could recreate the problem with was this:

var graphics;

function setup() {
    graphics = createGraphics(1000,1000);
}

function draw() {
    graphics.get();
}

This produces the expected result, but uses a ton of memory. Here's the relevant info I could get out of Chrome dev tools: The memory is currently at 1.2 gigs, but I've seen it reach 3 gigs, given enought time. Either way, that's way too much memory.

The saw tooth pattern continues indefinitely, with the teeth progressively growing slightly larger, but always dropping back down to the same height. I also checked out the javascript heap snapshot, but it rarely exceeded 10mb, even during peaks.

Based on this, my current theory is that the Graphics.get() method creates excess detached DOM nodes, which build up until garbage collected. However, I don't know why the spikes get progressively larger, nor do I have any idea as to how to fix it. If this is a known bug, or common mistake, any further information, solutions, or workarounds would be greatly appreciated. Thank you again so much for your help!

I have this, i've been able to display the pixel array only if the canvas is the same size as the image

`

var wH = window.innerHeight;
var wW = window.innerWidth;
var img;

function preload() {
  img = loadImage('assets/img.png');  
}

function setup() {
  var heigthImg = img.width/img.height*height
  var widthImg = img.height/img.width*width  
  createCanvas(wW, wH);
  background(51); 
  image(img, width/2 - heigthImg/2, 0, heigthImg ,wH);  

  //img.loadPixels();  
}

function draw() {  

//Displays image only if canvas is the same size as the image
//
//  loadPixels();
//   for (var y = 0; y < logoImg.height; y++) {
//    for (var x = 0; x < logoImg.width; x++ ) {
//      //Value index [r, g, b, a] pixel array
//      var pix = (x + y * logoImg.width)*4;
//      var r = logoImg.pixels[pix+0];
//      var g = logoImg.pixels[pix+1];
//      var b = logoImg.pixels[pix+2];
//      var a = logoImg.pixels[pix+3];


//      //set pixels
//      pixels[pix+0] = r;
//      pixels[pix+1] = g;
//      pixels[pix+2] = b;
//      pixels[pix+3] = a;
//    }
//  }
//  updatePixels();
}

`